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April  28, 1993 

The  Honorable  John  P.  Murtha 
Chairman,  Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 

Dear  Mr.  Chairman: 


As  requested,  we  examined  the  status  of  the  $9-bill»on  .Joint  Surveillance 
Target  Attack  Radar  System  (Joint  stars)  acquisition  program.  In  1993,  the 
Air  Force  and  Army  plan  to  seek  Department  of  Defease  (non)  approvals 
for  low-rate  initial  production  of  the  air  and  ground  segments  for  Joint 
stars.  The  full-rate  production  decisions  are  planned  for  1995  and  1996, 
respectively.  These  production  plans  have  raised  questions  concerning  the 
data  needed  to  permit  an  informed  full-rate  production  decision.  A  key 
issue  is  whether  the  1985  cost  and  operational  effectiveness  analysis 
(coea)  for  Joint  stake  is  sufficient  or  should  be  updated  or  replaced  by  a 
new  analysis. 


Results  in  Brief 


The  1985  coea  for  the  Joint  stars  is  no  longer  valid.  Significant  changes  in 
key  Joint  stars  concepts  such  as  the  threats  it  is  intended  to  meet,  the 
environment  in  which  it  will  operate,  and  what  it  will  cost  have  occurred. 
An  updated  or  new  coea  could  require  as  long  as  18  months  to  complete 
depending  on  its  scope.  As  currently  planned,  the  need  for  and  scope  of 
such  an  analysis  may  not  be  determined  until  6  months  prior  to  the 
full-rate  production  decision  scheduled  for  September  1995.  Waiting  that 
late  for  a  decision  on  a  coea  could  jeopardize  the  full-rate  production 
schedules  if  a  full  18-month  study  is  required.  Deciding  now  on  the  scope 
of  a  coea  would  eliminate  this  concern. 


Background 


Joint  stars  is  a  joint  Air  Force  and  Army  wide-area  surveillance  and  target 
attack  radar  system  being  designed  to  detect,  track,  classify,  and  support 
the  attack  of  moving  and  stationary  ground  targets.  The  program  consists 
of  air  and  ground  segments:  refurbished  707  aircraft  (designated  the 
E-8)  equipped  with  radar,  operation  and  control,  data  processing,  and 
communications  subsystems,  as  well  as  ground  stations  equipped  with 
communications  and  data  processing  subsystems. 
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The  program  is  currently  in  the  engineering  and  manufacturing 
development  phase  of  the  acquisition  cycle.  The  planned  initial  force 
structure  is  20  aircraft  and  95  ground  stations.  Program  acquisition  cost 
estimates  are  $7.4  billion  for  the  air  segment  and  $1.7  billion  for  the 
ground  segment.  Further  details  on  Joint  stars  program  production 
quantities  and  fiscal  year  1993  funding  levels  are  contained  in  appendix  I. 

In  1991,  during  Operation  Desert  Shield/Storm,  two  developmental 
E-8  aircraft  and  six  interim  ground  stations  were  deployed.  The  system 
provided  air  and  ground  components  with  49  consecutive  nights  of 
operational  support,  including  real-time  surveillance  and  targeting.  Air 
Force  officials  assessed  the  system’s  performance  as  excellent  and 
concluded  that  the  developmental  Joint  stars  made  a  significant 
contribution  to  eliminating  enemy  sanctuaries  and  providing  responsive 
targeting  and  intelligence.  Army  officials  also  assessed  the  system’s 
performance  as  excellent  and  concluded  that  the  developmental  Joint 
stars  was  the  single  most  valuable  intelligence  and  targeting  system 
supporting  Operation  Desert  Storm. 


Joint  STARS  Cost  and 
Operational 
Effectiveness  Analysis 
Is  No  Longer  Valid 


dod  regulations  require  that  coras  be  developed  and  considered  at  various 
decision  points  in  the  development  of  acquisition  programs.  The 
regulations  also  indicate  that  updated  or  new  coras  should  be  developed 
when  conditions  change.  Significant  changes  have  occurred  in  key 
concepts  affecting  Joint  stars  since  a  cora  was  developed  for  the  system 
in  1985.  Changes  in  threat,  operations,  and  costs  mean  that  an  updated  or 
new  Cora  should  be  developed  to  support  the  full-rate  production 
decisions. 


dod  Instruction  5000.2  requires  that  coras  be  prepared  and  considered  at 
milestone  decision  reviews  beginning  with  concept  demonstration 
approval  (Milestone  I)  for  msqor  programs  like  Joint  stars.  These  analyses 
are  intended  to  accomplish  three  objectives:  (1)  aid  decision-making  by 
illuminating  the  relative  advantages  and  disadvantages  of  the  alternatives 
being  considered  and  showing  the  sensitivity  of  each  alternative  to 
possible  changes  in  key  assumptions  (e.g.,  the  threat)  or  variables  (e.g., 
selected  performance  capabilities);  (2)  facilitate  communications  by  early 
identification  and  discussion  of  reasonable  alternatives  among 
decisionmakers  and  staffs  at  all  levels;  and  (3)  document  acquisition 
decisions  by  providing  the  analytical  underpinning  or  rationale  for 
decisions  on  a  program. 
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The  scope  of  a  coea  depends  upon  the  milestone  decision  to  be  made  and 
the  system’s  cost.  With  reference  to  Milestone  III,  production  approval,  the 
guidance  indicates  that  the  analysis  may  only  have  to  be  an  update  of  the 
Milestone  II  analysis.  The  guidance  also  states  that  an  assessment  is  not 
required  “unless  conditions  have  changed  sufficiently  so  that  previous  cost 
effectiveness  determinations  are  no  longer  valid.”  A  key  determination 
regarding  the  scope  of  a  new  Joint  stars  coea  then  would  be  the  extent  to 
which  conditions  have  changed  since  the  1985  Joint  stars  Milestone  13 
coea.  Assistant  Secretary  of  Defense  for  Program  Analysis  and  Evaluation 
officials  believe  that  the  scopes  of  past  analyses,  including  the  1985  coea, 
and  the  Desert  Storm  experience  studies  have  significant  limitations.  They 
also  believe  that  the  coea  should  be  updated  to  reflect  changes  in  threat 
and  missions,  new  performance  assumptions,  and  increased  costs. 

dod  guidance  on  how  to  do  a  coea  lists  a  dozen  key  concepts  that  are 
relevant  to  such  an  analysis.  We  reviewed  these  key  concepts  and 
concluded  that  most  of  the  concepts  or  conditions  affecting  them  have 
changed  substantially  since  the  1985  Milestone  II  coea.  Our  analysis  of 
these  key  concepts  is  presented  in  appendix  II  and  summarized  in  table  1. 


Table  1:  Substantial  Changes  in  Joint 
STARS  Cost  and  Operational  Concepts 
Since  1985 


Key  concept 

Substantial  change 

Yes  No  Partly 

Mission  needs,  deficiencies,  and  opportunities 

X 

Threats 

X 

Operational  environments 

X 

Constraints  and  assumptions 

X 

Operational  concept 

X 

Functional  objectives 

X 

Alternatives 

X 

Models 

X 

Data  for  analysis 

X 

Measures  of  effectiveness 

X 

Costs 

X 

Trade-of*  analyses 

X 

These  changes  in  key  concepts  could  ultimately  atiect  the  Joint  stars  force 
structure.  For  example,  changes  in  the  constraints  and  assumptions 
concept  raises  the  issue  of  affordability.  Joint  stars  entered  the 
engineeiing  and  manufacturing  development  phase  in  1985  during  a  period 
of  general  growth  in  the  overall  funding  of  the  defense  budget  The 


Page  3 


GAO/NSIAD-93-117  Tactical  Intelligence 


B-2519S4 


program  will  enter  the  production  phase  during  a  period  of  general 
decline.  In  addition,  evolving  Army  battlefield  intelligence  operational 
concepts  describe  a  synergistic  “system  of  systems”  in  which  Joint  stars 
will  play  an  important  role.  As  these  concepts  mature,  more  or  less  Joint 
stars  platforms  may  be  needed  to  complement  other  expected  battlefield 
intelligence  systems. 


The  Defense  Acquisition  Board  is  scheduled  to  begin  its  formal  1995 
full-rate  production  milestone  review  process  with  a  planning  meeting  in 
March  1995,  about  6  months  prior  to  the  milestone  review  itself.  One 
objective  of  the  planning  meeting  is  to  assess  the  plans  for  key  milestone 
documents  such  as  the  cost  estimate,  acquisition  strategy,  test  and 
evaluation  master  plan,  and  coea.  The  product  of  the  planning  meeting  will 
include  a  recommendation  on  whether  to  proceed  with  the  milestone 
review. 

Currently,  it  is  unclear  what  the  Defense  Acquisition  Board  may  require  in 
terms  of  a  Joint  stars  coea.  An  Under  Secretary  of  Defense  for  Acquisition 
representative  acknowledged  that  the  need  for  a  coea  to  support  the 
full-rate  production  decision  has  not  yet  been  resolved.  Army  officials 
estimate  that  a  comprehensive  coea  could  take  as  long  as  18  months.  The 
Army  estimate  suggests  that,  if  a  comprehensive  analysis  is  required,  work 
should  begin  no  later  than  March  1994  in  order  for  the  results  to  be 
available  to  support  the  air  segment’s  September  1995  full-rate  production 
decision. 

If  the  scope  of  the  coea  is  not  established  until  the  time  of  the  full-rate 
production  decision  planning  meeting  scheduled  for  March  1995  and  a 
comprehensive  18-month  analysis  is  determined  to  be  required,  the  Joint 
stars  production  schedule  could  be  placed  in  jeopardy. 


Timely  Cost  and 
Operational 
Effectiveness  Analysis 
Decision  Is  Critical 


Recommendations 


We  recommend  that  the  Secretary  of  Defense  direct  the  Under  Secretary 
of  Defense  for  Acquisition  to  perform  an  updated  or  new  coea  to  support 
full-rate  production  of  Joint  stars. 

We  also  recommend  that  the  Secretary  direct  that  the  Under  Secretary 
determine  the  scope  of  the  coea  by  March  1994  to  allow  adequate  time  for 
the  evaluation  without  affecting  the  Joint  stars  acquisition  plan. 
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Agency  Comments 


dod  agreed  or  partially  agreed  with  the  findings  and  recommendations  in 
this  report,  dod  stated  that  a  May  1993  Defense  Acquisition  Board  will 
review  the  need  for  updating  the  1985  coea  for  Joint  stars.  Also,  direction 
to  the  Army  and  the  Air  Force  to  perform  further  cost  and  operational 
effectiveness  analysis  is  expected  to  be  provided  in  an  acquisition  decision 
memorandum  following  the  Defense  Acquisition  Board  review.  In 
addition,  dod  commented  that  whatever  analyses  are  determined 
necessary  would  be  completed  in  time  to  avoid  adversely  affecting  the 
Joint  stars  acquisition  plan,  dod’s  plan  to  have  the  May  Defense 
Acquisition  Board  reviews  of  the  E-8  aircraft  and  the  ground  station 
determine  what  additional  analysis  is  needed  is  consistent  with  our 
recommendations.  We  will  continue  to  monitor  dod,  Army,  and  Air  Force 
actions  to  implement  our  recommendations. 

dod’s  comments  and  our  responses  are  included  in  appendix  III. 


Scope  and 
Methodology 


We  interviewed  program  officials  and  examined  program  management  and 
budget  documents,  system  requirements,  test  plans  and  results,  acquisition 
plans  and  schedules,  and  other  program  documentation.  We  performed 
work  at  the  Air  Force  Air  Combat  Command,  Langley  Air  Force  Base, 
Virginia;  Air  Force  Electronic  Systems  Center,  Hanscom  Air  Force  Base, 
Massachusetts;  Army  Training  and  Doctrine  Command,  Fort  Monroe, 
Virginia;  and  Army  Communications  and  Electronics  Command,  Fort 
Monmouth,  New  Jersey.  We  also  contacted  program  representatives 
within  the  Office  of  the  Assistant  Secretary  of  Defense  for  Program 
Analysis  and  Evaluation;  Director,  Operational  Test  and  Evaluation;  and 
the  Departments  of  the  Air  Force  and  Army. 

We  performed  our  review  from  January  1992  to  March  1993  in  accordance 
with  generally  accepted  government  auditing  standards. 


Unless  you  publicly  announce  its  contents  earlier,  we  plan  no  further 
distribution  of  this  report  until  10  days  after  its  issue  date.  At  that  time,  we 
will  send  copies  to  other  interested  congressional  committees;  the 
Secretaries  of  Defense,  the  Army,  and  the  Air  Force;  the  Director,  Office  of 
Management  and  Budget;  and  other  interested  parties.  We  will  make 
copies  available  to  others  upon  request. 
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Please  contact  me  on  (202)  512-4841,  if  you  or  your  staff  have  any 
questions  concerning  this  report.  Other  major  contributors  are  listed  in 
appendix  IV. 


Director,  Systems  Development 
and  Production  Issues 
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Joint  STARS  Program  Production  Quantities 
and  Fiscal  Year  1993  Funding  Levels 


Air  Segment 

Table  1.1  shows  how  the  20  aircraft  will  be  acquired. 

1  8016  1.1 ,  JOlfU  9  1  AM9  MlfCi 8T1 

Acquisition 

Acquisition  phase 

Quantity 

Engineering  and  manufacturing  development 

3 

Low-rate  initial  production 

5 

Full-rate  production 

12 

Total 

20 

The  three  engineering  and  manufacturing  development  aircraft  are  under 
contract  and  in  va  ious  phases  of  testing  or  refurbishment. 

Fiscal  year  1993  funding  is  to  be  used  to  produce  the  first  two  low-rate 
initial  production  aircraft  and  to  acquire  advance  procurement  items  for 
both  the  third  and  fourth  low-rate  initial  production  aircraft.  Table  1.2 
summarizes  the  outcome  of  the  fiscal  year  1993  funding  process. 


Table  1.2:  Fiscal  Year  1993  Air  Segment 
Funding 

Dollars  in  millions 

Purpose 

Budget 

request 

Quantity 

Funding 

Quantity 

Engineering  and 
manufacturing  development 

$355.9 

a 

$333.4 

4 

Low-rate  initial  production 
procurement 

310.6 

i 

5118 

2 

Advance  procurement 

50.7 

i 

79.1 

2 

•Funding  for  testing  and  refurbishment  of  previously  procured  aircraft. 

Funding  for  the  remaining  low-rate  initial  production  aircraft  will  be 
requested  in  fiscal  years  1994  and  1995. 


Ground  Segment 


Table  1.3  shows  the  Army’s  planned  ground  station  production 
configuration. 


Table  1.3:  Ground  Station  Production 
Configuration 

Block 

Configuration 

Quantity 

1  Medium 

5-ton  truck 

12 

1  Light 

Highly  mobile  multipurpose  wheeled  vehicle 

45 

1  Heavy 

Electronic  fighting  vehicle 

7 

II 

Common  ground  station 

31 

Total 

95 
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Joint  STARS  Program  Production  Quantities 
and  Fiscal  Year  1993  Funding  Level* 


According  to  dod,  the  validated  number  of  ground  stations  is  75.  The  Joint 
Requirements  Oversight  Council  will  have  to  review  the  Army’s  plans  to 
increase  the  number  of  ground  stations  to  95.  The  fiscal  year  1993  budget 
requested  funding  to  continue  engineering  and  manufacturing 
development  efforts  ($31.2  million)  and  to  produce  the  first  five  Block  1 
Medium  ground  stations  ($36.2  million).  The  Congress  provided  the  funds 
requested  and  provided  increased  funding  to  accelerate  the  engineering 
and  manufacturing  development  efforts.  Table  1.4  summarizes  the 
outcome  of  the  fiscal  year  1993  funding  process. 


Segment  Funding  Dollars  in  millions 

Purpose 

Budge* 

request 

Quantity 

Funding 

Quantity 

Engineering  and 
manufacturing  development 

$31.2 

a 

$66.2 

a 

Block  1  Medium  production 

36.2 

5 

36.2 

5 

“Funding  for  testing  and  development  of  ground  stations. 


Funding  for  the  remaining  seven  Block  I  Medium  ground  stations 
($54.5  million)  will  be  included  in  the  fiscal  year  1994  budget  request. 
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GAO  Analysis  of  Changes  in  Joint  STARS 
Cost  and  Operational  Concepts  Since  1985 


Key  concept 

Substantial 

change 

GAO  analysis 

Mission  needs, 
deficiencies,  and 
opportunities 

No 

According  to  DOO  guidance,  this  type  of 
analysis  is  conducted  for  Milestone  1. 
concept  demonstration  approval,  and 
Milestone  II,  development  approval. 

As  they  did  in  1985,  the  Air  Force  and  Army 
continue  to  articulate  a  need  for  near-real¬ 
time,  wide-area  surveillance  of  moving 
targets  on  the  battlefield.  Army  officials  are 
also  adamant  in  articulating  a  need  for  a 
system  to  replace  the  Mohawk  surveillance 
system,  which  is  estimated  to  end 
operational  support  to  Army  corps  by  1996. 

In  addition,  Army  officials  believe  that  new 
battiefiela  strategies-— win  fast,  win 
decisively,  with  minimum  casualties— 
emphasize  the  need  for  systems  like  Joint 
STARS 

Threats 

Yes 

It  is  clear  that  the  main  threat  envisioned  in 
1985— Soviet  Union  and  Warsaw  Pact 
forces— has  reduced  substantially 

Operational  environments 

No 

The  physical  operational  environment  has 
not  changed  Joint  STARS  is  designed  to  be 
a  "near  all-weather"  system 

Yes 

The  Joint  STARS  survivability  requirements 
have  evolved  since  1985 

Constraints  and 
assumptions 

Yes 

The  1985  COEA  did  not  identify  a  concern 
about  the  affordability  of  the  program  Joint 
STARS  entered  the  engineering  and 
manufacturing  development  phase  in  a 
period  of  general  growth  in  the  overall 
funding  of  the  defense  budget  The  program 
will  enter  the  production  phase  during  a 
period  of  general  decline  in  the  overall 
funding  of  the  defense  budget 

(continued) 
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GAO  Analyst*  of  Change*  in  Joint  STABS 
Coat  and  Operational  Concept*  Since  1986 


Key  concept 

Substantial 

change 

GAO  analysis 

Operational  concept 

Yes 

These  plans  describe  the  way  in  which 
forces  and  equipment  would  be  arranged 
and  employed  in  battle.  Sometimes,  field 
experimentation  is  necessary  to  refine  a 
plan.  The  1985  COEA  identified  a  single 
joint  operational  and  organizational  plan.  In 
1990,  the  Air  Force  and  Army  published  a 
joint  concept  of  operations  that  would  also 
serve  as  the  operational  reference  for  the 
development  of  three  theater-specific 
operational  plans  and  concepts. 

Joint  STARS'  testing,  operational  field 
demonstrations,  and  the  Desert  Storm 
experience  have  provided  an  abundance  of 
operational  data  since  1985  that  can  be 
used  to  validate  operational  concepts  for 
Joint  STARS.  For  example,  user/operator 
comments  from  an  Army  Operational 
Evaluation  Command  summary  report  on 
Operation  Desert  Storm  reflect  different 
opinions  on  the  utility  of  Joint  STARS  as  a 
corps  versus  theater  asset  and  continuous 
surveillance  versus  selected  missions.  After 
continued  analysis,  these  data  could  affect 
the  number  of  Joint  STARS  orbits  needed  or 
the  number  of  aircraft  needed  per  orbit. 

Functional  objectives 

No 

Functional  objectives  are  statements 
describing,  in  quantitative  terms,  the  tasks  a 
system  will  be  expected  to  perform.  The 

Joint  STARS  functional  objectives  do  not 
appear  to  have  changed  substantially  since 
1985. 

Alternatives 

No 

Although  the  1985  Joint  STARS  COEA  did 
not  examine  alternative  systems,  the  1987 
Operational  Utility  Evaluation  assessed  the 
survivability,  operational  utility,  and  cost  of 
Joint  STARS  and  alternative  systems  of 
air/land  battle  management,  including 
situation  and  target  development  and  target 
attack.  The  1987  evaluation  sunported  the 
Milestone  IIB  decision.  Given  that  the 
objective  of  the  program  is  to  provide  near- 
real-time,  wide-area  surveillance  of  the 
battlefield  by  1997,  it  does  not  appear  that 
there  is  any  single  system  that  is  a  viable 
alternative  to  Joint  STARS. 

(continued) 
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GAO  Anxlyai*  of  Change*  in  Joint  STARS 
Coat  and  Operational  Concept*  Since  1985 


Key  concept 

Substantial 

change 

GAO  analysis 

Models 

Yes 

The  effectiveness  and  utility  of  the  Joint 
STARS  radar  reported  in  the  1985  COEA 
was  derived  from  models  using  only  moving 
target  indicator  radar  data,  whereas,  the 

Joint  STARS  requirement  is  for  a  multimode 
radar  The  abundance  of  operational  data 
gathered  since  1985  would  also  enhance 
any  new  modeling 

Data  for  analysis 

Yes 

There  is  an  abundance  ot  new  operational 
data  available  to  analysts 

Measures  of  effectiveness 

Yes 

Although  the  1985  COEA  identified 
operational  utility  as  a  critical  issue  and 
developed  measures  of  performance  for  use 
in  the  analysis,  no  measures  of  effectiveness 
were  identified.  These  are  currently  being 
developed  for  the  Air  Force  and  Army 
segments  by  the  services'  operational  test 
and  evaluation  centers  for  use  during  the 
Multi-Service  Operational  Test  and 
Evaluation.  That  test  and  evaluation  will  be 
used  to  support  the  full-rate  production 
decision  in  1995.  DOD's  Director  of 
Operational  Test  and  Evaluation  is 
monitoring  the  development  of  these 
measures. 

Costs 

Yes 

The  1985  COEA  used  fiscal  year  1983 
dollars  to  estimate  a  unit  cost  of 
$151.5  million  for  24  C-18  airborne  systems. 
In  December  1991.  a  comparable  unit  cost 
estimate  was  $235.4  million — an  increase  of 
55  percent—  for  20  used  707  aircraft. 

In  addition,  in  1985,  the  approved  force 
structure  was  10  C-18s.  Currently,  the  initial 
force  structure  is  projected  to  be  20 
refurbished  707s 

Trade-off  analyses 

No 

These  analyses  are  an  important  component 
of  both  Milestone  1  and  II  analyses  To  do  a 
trade-off  analysis,  one  must  identify  areas  of 
uncertainty,  conduct  sensitivity  analyses, 
and  establish  thresholds. 

Yes 

Performance  thresholds  show  at  what  point 
degradations  in  performance  yield 
outcomes  that  no  longer  satisfy  the  mission 
need  In  November  1991,  the  Air  Force  and 
Army  Chiefs  of  Staff  performed  a  review  of 
the  operating  command  requirements  and 
status  of  the  program  in  meeting  those 
requirements.  As  a  result,  29  thresholds  for 
noncritical  characteristics  were  replaced 
with  objective  values,  and  9  critical 
characteristics  were  clarified,  with  clearly 
stated  thresholds  being  established. 
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Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


See  comment  1 . 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

WASHINGTON,  DC  20301  -3000 


o  5  t*AR  'ft® 


Mr.  Frank  C.  Conahan 
Assistant  Comptroller  General 
National  Security  and  International 
Affairs  Division 

U.  S.  General  Accounting  Office 
Washington,  D.C.  20648 

Dear  Mr.  Conahan: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  General  Accounting  Office 
(GAO)  draft  report  entitted-TACTICAL  INTELLIGENCE:  Joint  STARS  Needs  Current  Cost 
and  Operational  Effectiveness  Analysis,'  dated  January  26,1993  (GAO  Code  3951 81/OSD 
Case  93 12)  The  Department  partially  concurs  with  the  report 

The  DoO  agrees  that  significant  changes  have  occurred  in  the  European  theater  since 
the  initial  Joint  Surveillance  and  Target  Attack  Radar  System  Cost  and  Operational 
Effectiveness  AnatysiswaB  accomplished  in  1985.  However,  such  changes  do  net  render 
useless  all  of  the  analyses  performed  and  the  information  derived  from  combat  operations 
since  then.  The  Department  is  currency  reviewing  the  need  for  updating  the  Joint  Surveillance 
and  Target  Attack  Radar  System  Cost  and  Operational  Effectiveness  Analysis,  and  will 
decide  on  the  matter  after  the  Defense  Acquisition  Board's  review  of  the  low  rate  initial 
production  for  the  E-8C  and  the  Ground  Station  Module  Block  I  Medium,  scheduled  for 
May  1993. 

The  Department  partially  concurs  with  the  recommendation  that  the  Under  Secretary 
of  Defense  for  Acquisition  direct  that  an  updated  or  new  Cost  and  Operational  Effectiveness 
Analysis  be  completed  to  support  the  production  milestone  decision  To  do  so  at  this  time  is 
premature  in  view  of  the  upcoming  Defense  Acquisition  Board  review.  The  Department 
concurs  with  the  recommendation  to  make  a  decision  on  any  updated  analysis  by  March  1994 
A  decision  on  the  matter  will  be  made  weti  in  advance  to  allow  adequate  time  for  an  updated  or 
new  cost  and  operational  effectiveness  analysis  to  be  accomplished  prior  to  the  production 
milestone. 

The  detailed  DoO  comments  on  the  report  findings  and  recommendations  are  provided 
m  the  enclosure.  The  DoO  appreciates  the  opportunity  to  comment  on  the  GAO  draft  report 

Sincerely, 

Frank  Kendall 

Director 

Tactical  Systems 


ACQUISITION 
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Nowon  pp.  1-2. 


GENERALACCOUNTMG  OFFICE  DRAFT  REPORT -DATED  JANUARY  26, 1893 
(GAO  CODE  385181)  OSD  CASE  8312 

TACTICAL  NTELLJGENCE:  JOMT  STARS  NEEDS  CURRENT  COST  AND  OPERATIONAL 
EFFECT  IVENESS  ANALYSIS' 

DEPARTMENT  OF  DEFENSE  COMMENTS 


FMDMGS 


•  TTlT^!VTrff»K»T*rg*<Att»cK Radar $y*»m  Psrtamanc*  During 
OPERATION  DESERT  STORM.  The  GAO  reported  that  the  Joint  Surveillance  Target 
Attack  Radar  System  is  a  joint  Air  Force  and  Army  long-range,  wide-area  surveillance 
and  target  attack  radar  system -and  a  $9  biSton  acquisition  program.  The  GAO 
explained  that  the  program  is  currently  in  the  engineering  and  manufacturing  develop¬ 
ment  phase  o(  the  acquisition  cyde.  The  GAO  observed  that  planned  initial  force 
structure  is  20  aircraft  and  95  ground  stations,  with  estimated  costs  of  $7  4  billion  for 
the  ax  segment  and  $1.5  biion  for  the  ground  segment  The  GAO  reported  lhat  during 
OPERATION  DESERT  STORM,  two  developmental  E-8  aircraft  and  six  interim  ground 
stations  were  deployed,  and  that  system  provided  air  and  ground  components  with  *19 
consecutive  nights  of  operational  support,  including  real  time  surveillance  and 
targeting.  The  GAO  noted  that  Air  Force  officials  assessed  the  system  performance 
as  excellent  determining  that  the  developmental  Joint  Surveillance  Target  Attack 
Radar  System  made  a  significant  contrftxrtion  to  eliminating  enemy  sanctuaries  and 
providing  responsive  targeting  and  intelligence.  The  GAO  further  noted  that  Army 
officials  also  assessed  system  performance  as  excellent,  and  determined  that  the 
developmental  Joint  Surveillance  Target  Attack  Radar  System  was  the  single  most 
valuable  intettgence  and  targeting  system  supporting  OPERATION  DESERT  STORM, 
(pp.  1-3/GAO  Draft  Report) 

Pop  RESPONSE:  Conor. 

*  FINDING  B:  Joint  SurysWance  Target  Attack  Radar  System  Cost  and  Oper^^ 
Eftacftvsnses  Analyst*  >t  No  Longer  Vs8d  The  GAO  reported  that  DoD  Instruction 
5000.2  requires  that  Cost  and  Operational  Effectiveness  Analyses  be  developed  and 
considered  at  various  decision  points  in  the  development  of  weapon  systems,  and  that 
new  analyses  be  developed  when  ccndttons  have  changed  The  GAO  found  that 
significant  changes  have  occurred  in  key  concepts  affecting  the  Joint  Surveillance 
Target  Attack  Radar  System  since  a  Cost  and  Operational  Effectiveness  Analysis  was 
developed  for  the  system  in  1985.  The  GAO  concluded,  therefore,  that  changes  in 
threat  operations,  and  costs  mean  that  an  updated  or  new  Cost  and  Operational 
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Now  on  pp.  2-4. 


See  comment  2. 


See  comment  1 . 


See  comment  3. 


Effectiveness  Analysis  should  be  developed  to  support  the  full-rate  production 
decisions. 

The  GAO  observed  that  the  scope  of  a  Cost  and  Operational  Effectiveness  Analysis 
depends  on  the  milestone  decision  to  be  made  and  the  system  cost  The  GAO  explained 
that  DoD  guidance  on  how  to  do  a  Cost  and  Operational  Effectiveness  analysts  lists  a 
dozen  key  concepts  that  are  relevant  to  such  an  analysis.  The  GAO  found  that  most  of 
the  key  concepts  and  conditions  related  to  the  system  have  changed  substantially  since 
the  1985  Milestone  II  Cost  and  Operational  Effectiveness  Analysis.  The  GAO  noted,  for 
example,  that  affordability  is  an  issue,  since  the  system  will  enter  the  production  phase 
during  a  period  of  general  budget  dedine,  while  the  system  entered  the  engineering  and 
manufacturing  development  phase  in  1985,  during  a  period  of  general  growth  in  the 
overall  funding  of  the  defense  budget.  The  GAO  also  observed  that  as  operational 
concepts  mature  -  such  as  the  evolving  Army  battlefield  intelligence  synergistic 
'system  to  systems'  -  more  or  fewer  Joint  Surveillance  Target  Attack  Radar  System 
platforms  may  be  needed  to  complement  other  expected  battlefield  intelligence 
systems.  The  GAO  concluded  that  changes  in  key  concepts  ultimately  could  affect  the 
system  force  structure,  (pp.  4-7/GAO  Draft  Report) 

DoO  RESPONSE  Partially  Concur.  The  DoD  agrees  that,  because  of  the  significant 
changes  in  the  threat  the  analytical  rationale  for  the  E-8  and  the  Ground  Station 
Module  may  have  to  be  updated.  However,  the  change  in  the  threat  is  not  as  drastic  as 
the  GAO  has  observed.  The  current  System  Threat  Assessment  Report  for  the  Joint 
Surveillance  and  Target  Attack  Radar  System  program  indicates  that,  although  the 
primary  theater  of  operation  envisioned  for  the  system  has  changed,  the  threat  is 
composed  of  the  same  type  of  opposition  force  that  has  existed  for  the  last  decade, 
trained  in  the  same  tactics  and  operations.  The  total  number  of  forces  involved  at  any 
one  time  may  have  shifted  with  the  breakup  of  the  Warsaw  Pact  and  Soviet  Union,  but 
the  tanks  and  the  planes  are,  for  all  purposes,  still  there.  Fa  example,  DESERT 
STORM  prcMdes  the  example  of  a  massed  armor  enemy  while  Kaea  remains 
unchanged  as  a  Soviet  style  offensive  force. 

The  DoD  does  not  agree  that  all  the  oost  and  operational  effectiveness  related 
analyses  are  no  longer  valid.  Furthermae,  a  process  is  in  place  to  determine  at  the  low 
rate  initial  production  Defense  Acquisition  Beard  reviews  in  May  1 993  for  the  E-8  and 
the  Ground  Station  Module  programs,  what  further  analyses  may  be  needed  for  the 
production  decision  reviews  in  1 995.  Following  the  Gulf  War,  the  Conventional  Systems 
Committee  reviewed  the  Joint  Surveillance  and  Target  Attack  Radar  System  program 
to  determine  the  impacts  of  the  deployment  of  the  two  test  aircraft,  ground  station 
modules,  and  supporting  personnel  and  equipment  to  DESERT  STORM.  The  Office  of 
the  Secretary  of  Defense  questioned  the  validity  of  the  existing  cost  and  operational 
effectiveness  documentation  and  asked  the  Army  and  the  Air  Face  to  provide  after 
action  reports,  analyses,  and  lessons  learned  that  came  out  of  the  DESERT  STORM 
experience  pertaining  to  the  Joint  Surveillance  and  Target  Attack  Radar  System.  Those 
reports  were  subsequently  developed.  In  addition,  since  the  Conventional  Systems 
Committee  review  of  the  E-8  long  lead  items  in  January  1992,  the  Army  and  the  Air 
Force  have  been  engaged  in  a  process  to  provide  updated  analytical  ration -te  and  to 
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See  comment  1. 
See  comment  3. 


See  comment  4. 
See  comment  5. 


Now  on  p.  4. 


See  comment  3. 


identify,  for  the  Defense  Acquisition  Board  in  May  1 993,  the  remaining  issues  and  areas 
that  require  additional  analyses. 

The  GAO  report  also  indcaies  substantial  change  has  occurred  in  the  operational 
concept  for  the  Joint  Surveillance  and  Target  Attack  Radar  System  The  Joint 
Requirements  Oversight  Council  has  reviewed  the  concept  of  operation  and  has 
validated  a  force  structure  requirement  for  20  E-8C  aircraft  and  75  ground  station 
modules.  The  Joint  Requirements  Oversight  Council  will  review  the  Army  proposal  to 
increase  the  ground  station  module  requirement,  prior  to  the  next  Defense  Acquisition 
Board  Milestone  review.  Regarcing  the  operational  concept  decisions  by  a  theater 
commander-in-chief  to  employ  scarce  resources  (E-8A  and  interim  ground  station 
modules),  in  a  nan-doctrinal  way,  do  not  change  the  operational  concept  There  ha^e 
been  no  changes  in  the  operational  concept 


FINDING  C  Tlrnefr  Cost  and  Operational  EfTectivenessAfugyslsDeciskm  to 

The  GAO  reported  that  the  Defense  Acquisition  Board  is  scheduled  to  begin  its  formal 
1 995  full-rate  production  milestone  review  process  with  a  planning  meeting  in  March 
1995,  The  GAO  explained  that  it  is  unclear  what  the  Defense  Acquisition  Board  may 
require  in  terms  of  a  Joint  Surveillance  T arget  Attack  Radar  System  Cost  and 
Operational  Effectiveness  Analysis.  The  GAO  noted  that  according  to  an  official 
within  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  the  need  for  a 
Cost  and  Operational  Effectiveness  Analysis  to  support  the  full-rate  production 
decision  has  not  yet  been  resolved  The  GAO  further  noted  that  according  to  Army 
officials,  a  comprehensive  Cost  and  Operational  Effectiveness  Analysis  could  take  as 
tongas  18  months.  The  GAO  concluded  that  based  on  the  Army  estimate,  if  a 
comprehensive  analysis  is  required,  work  should  begin  no  later  than  March  1994,  in  order 
for  the  results  to  be  available  to  support  the  air  segment  September  1995  full-rate 
production  decision.  The  GAO  further  concluded  that,  if  the  scope  of  the  Cost  and 
Operational  Effectiveness  Analysis  for  the  Joint  Surveillance  Target  Attack  Radar 
System  is  not  established  until  the  time  of  the  full-rate  production  decision  planning 
meeting,  scheduled  tar  March  1995,  and  a  comprehensive  18-month  analysis  is 
determined  to  be  required,  the  production  schedule  could  be  plaoed  in  jeopardy.  The 
GAO  also  noted  that  one  resulting  outcome  might  a  break  in  Joint  Surveillance  Target 
Attack  Radar  System  production,  with  consequent  increased  production  costs,  (pp. 
7-8/GAO  Draft  Report) 

DoO  RESPONSE  Concur.  The  determination  of  the  need  for  an  updated  or  new  Cost 
and  Operational  Effectiveness  Analysis  will  occur  before  the  March  1 994  date  specified 
in  the  GAO  report  As  currently  planned,  in  May  1 993  the  Defense  Acquisition  Board 
will  review  the  need  for  updating  the  1985  Cost  and  Operational  Effectiveness  Analysis 
to  support  the  E-8  and  the  Ground  Station  Module  Milestone  111  Production  decision, 
which  is  scheduled  for  1 995.  An  expected  outcome  of  that  review  will  be  the  direction 
to  update  all  or  portions  of  the  analysis.  In  any  event  the  cost  aspects  of  both 
programs  are  being  updated  as  part  of  the  upcoming  May  1 993  Defense  Acquisition 
Board  review  for  low  rate  initial  production. 
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Now  on  p.  4. 


See  comment  3. 


RECOMMENDATIONS 


RECOMMENDATION:  The  GAO  recommended  that  the  Under  Secretary  of  Defense 
for  Acquisition  direct  that  an  updated  or  new  Cost  and  Operational  Effectiveness 
Analysis  be  completed  to  support  full-rate  production  of  the  Joint  Surveillance  Target 
Attack  Radar  System,  (p.  8/GAO  Draft  Report) 

PoO  RESPONSE:  Pattafty  Concur.  The  Defense  Acquisition  Board  wil  review  the 
need  for  updating  the  1 985  Cost  and  Operational  Effectiveness  Analysis  at  the  low 
rate  initial  production  reviews  in  May  1 993,  and  will  advise  the  Under  Secretary  of 
Defense  for  Acquisition.  To  direct  an  updated  or  new  Cost  and  Operational  Effective¬ 
ness  Analysis*  prior  to  the  Board's  review  would  be  premature. 


Now  on  p.  4. 


See  comment  3. 


RECOMMENDATION:  The  GAO  recommended  that  the  Under  Secretory  of  Defense 
for  Acquisition  direct  that  the  scope  of  the  Cost  and  Operational  Effectiveness 
Analysis  be  determined  by  March  1994  -  in  order  to  allow  adequate  time  for  the 
evaluation,  without  impacting  the  Joint  Surveillance  Target  Attack  Radar  System 
acquisition  plan.  (p.  8/GAO  Draft  Report) 

DoO  RESPONSE:  Conor.  The  direction  to  the  Army  and  the  Air  Face  to  perform 
further  cost  and  operational  effectiveness  analysis  is  expected  to  be  provided  in  the 
acquisition  decision  memorandum,  following  the  upcoming  May  1993  review  by  foe 
Defense  Acquisition  Board.  Whatever  analyses  are  determined  necessary  wiB  be 
completed  in  time  to  permit  adequate  evaluation,  without  adversely  impacting  foe 
Joint  Surveillance  and  Target  Attack  System  acquisition  plan. 
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The  following  are  gao’s  comments  on  the  Department  of  Defense’s  letter 
dated  March  5,  1993. 


GAO  Comments 


1.  We  do  not  believe  that  changes  since  the  1985  coea  have  rendered 
interim  analyses  useless.  Rather,  we  believe  that  the  interim  analyses 
provide  additional  data  to  reevaluate  the  cost  and  operational 
effectiveness  of  the  system  and  enhance  any  new  modeling. 

2.  We  are  encouraged  that  dod  recognizes  that  because  of  significant 
changes  in  threat,  the  analytical  rationale  for  the  Joint  stars  may  have  to 
be  updated,  dod’s  1993  annual  report  to  the  Congress  states  that  the 
United  States  no  longer  faces  the  threat  of  global  war  beginning  in  Europe, 
but  still  faces  serious  regional  contingencies.  However,  the  report  also 
states  that  a  new  regional  strategy  means  that  the  force  structure  can  be 
reduced  and  reshaped.  For  example,  it  notes  that  efforts  are  underway  to 
eliminate  one-third  of  the  Army’s  active  divisions,  one-fifth  of  the  Navy’s 
ships,  and  10  Air  Force  fighter  wing  ec  ivalents.  The  forces  that  remain 
are  being  restructured  under  new  wamghting  doctrines.  Our  point  is  that 
the  threat  has  been  substantially  reduced,  resulting  in  reductions  in  U.S. 
forces.  These  impacts  should  be  considered  in  a  new  or  updated  coea 

3.  dod’s  plan  to  have  the  Defense  Acquisition  Board’s  reviews  of  die  E-8 
and  the  ground  station  determine  what  additional  analyses  are  needed  is 
consistent  with  our  recommendations.  We  will  continue  to  monitor  dod, 
Army,  and  Air  Force  actions  to  implement  our  recommendations. 

4.  A  February  1985  Joint  Services  Operational  Requirement  document 
noted  that  a  joint  system  operational  concept  would  be  developed  by  the 
Air  Force  and  Army  and  would  detail  all  aspects  of  joint  operations.  The 
May  1985  coea  referenced  a  draft  Joint  Systems  Operational  Concept  and 
identified  a  single  joint  operational  and  organizational  plan.  As  late  as 
November  1989,  the  U.S.  Air  Force  and  Army  in  Europe  were  working  to 
resolve  Joint  stars  use  issues  and  develop  a  more  detailed  concept  of 
operations.  In  1990,  the  Air  Force  and  Army  published  a  joint  concept  of 
operations  that  would  also  serve  as  the  operational  reference  for  the 
development  of  three  theater-specific  operational  plans  and  concepts. 

We  recognize  that  Desert  Storm  may  have  provided  an  opportunity  for  a 
nondoctrinal  employment  of  theater  assets  by  the  theater 
commander-in-chief.  However,  user/operator  comments  in  an  Army 
Operational  Evaluation  Command  summary  report  on  Operation  Desert 
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Storm  reflect  different  opinions  on  the  utility  of  Joint  stars  as  a  corps 
versus  theater  asset  and  continuous  surveillance  versus  selected  missions. 
After  continued  analysis,  these  data  could  affect  the  number  of  Joint  stars 
orbits  needed  or  the  number  of  aircraft  needed  per  orbit.  We  have  added 
data  to  appendix  II  to  reflect  these  comments. 

5.  We  have  added  data  to  explain  the  number  of  ground  stations. 
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